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Abstract 

Patient  Safety  Indicators  (PSIs),  developed  by  the  Agency  for  Healthcare 
Research  and  Quality  (AHRQ),  are  administrative  data-based  indicators  that 
identity  potential  in-hospital  patient  safety  events.  This  study  developed  and 
tested  methods  for  (1)  applying  PSIs  to  Department  of  Veterans  Affairs  (VA) 
discharge  data,  and  (2)  comparing  VA  with  non-VA  PSI  rates.  VA  inpatient  data 
file  structure  and  elements  were  modified  in  order  to  apply  PSIs  to  VA  data; 
further  modifications  were  required  to  compare  VA  and  non-VA  PSI  rates.  We 
found  that  key  measures,  including  demographics,  clinical  elements,  and  length  of 
stay,  as  well  as  the  PSI  rates  themselves,  are  sensitive  even  to  minor  data 
modifications.  This  paper  demonstrates  both  the  adaptation  of  a  database  for  use 
with  the  PSIs,  and  the  sensitivity  of  PSI  rates  to  small  differences  in  database 
characteristics.  The  paper  shows  how  differences  in  data  sources  might  affect 
comparisons  of  event  rates  across  health  care  systems. 


Introduction 

Patient  safety  has  become  a  national  priority.  However,  due  to  the  lack  of 
standardized  terminology  or  methodology  for  identifying  patient  safety  problems, 
the  rates  of  reported  patient  safety  events  vary  widely  in  the  literature.1  27  The 
lack  of  a  standard  method  is  problematic  for  a  number  of  reasons,  including  the 
fact  that  comparing  quality  of  care,  of  which  patient  safety  is  an  integral 
component,  requires  meaningful,  reliable,  and  valid  performance  measures  that 
can  be  used  across  health  care  systems  and  settings.  Thus,  development  of 
standardized  generic  tools  that  can  capture  potentially  preventable  patient  safety 
events  is  a  necessary,  though  challenging,  step  in  promoting  a  better 
understanding  of  the  magnitude  of  the  problem  and  in  furthering  the  development 
of  interventions  aimed  at  reducing  patient  safety  events. 

Patient  Safety  Indicators  and  their  development 

The  Patient  Safety  Indicators  (PSIs),  developed  by  the  Agency  for  Healthcare 
Research  and  Quality  (AHRQ)  and  revised  by  the  University  of  California  at  San 
Francisco-Stanford  University  Evidence-based  Practice  Center  (UCSF-Stanford 
EPC),  are  a  set  of  administrative  data-based  indicators  used  to  identify  potential 
in-hospital  patient  safety  events.28  The  AHRQ  PSIs  have  their  roots  in  the 
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Institute  of  Medicine’s  definition  of  patient  safety:  “freedom  from  accidental 
injury  caused  by  medical  care.”14  This  definition  has  since  been  expanded  to 
include  “the  failure  of  a  planned  action  to  be  completed  as  intended  or  the  use  of  a 
wrong  plan  to  achieve  an  aim.  Errors  can  include  problems  in  practice,  products, 
procedures,  and  systems.”24  The  PSIs  are  measured  as  rates  defined  as  outcome  of 
interest/population  at  risk.  For  example,  the  rate  of  the  hospital-level  PSI 
Complications  of  Anesthesia  is  the  number  of  discharges  with  this  complication, 
divided  by  the  total  number  of  surgical  discharges.  PSIs  track  surgical 
complications  and  other  iatrogenic  events,  screening  for  “potential  problems  that 
patients  experience  resulting  from  exposure  to  the  health  care  system,  and  that  are 
likely  amenable  to  prevention  by  changes  at  the  system  level.”28 

PSIs  represent  a  significant  advance  in  the  development  of  a  methodology  for 
identifying  patient  safety  events.  The  PSIs,  unlike  previous  measures  evaluating 
complications  or  adverse  events  related  to  hospitalization,  were  specifically 
developed  to  capture  those  instances  representing  potentially  preventable  adverse 
events  that  compromise  patient  safety  in  the  inpatient  setting,  such  as  surgical 
complications,  death  in  cases  with  low-mortality  diagnoses,  and  decubitus 
ulcers.  ’  In  this  paper,  we  focus  on  AHRQ’s  accepted  hospital-level  PSIs,  which 
were  developed  through  a  four-step  process  that  included  literature  review, 
evaluation  of  candidate  PSIs  by  clinical  panels,  expert  review  of  ICD-9-CM  codes 
in  candidate  PSIs,  and  empirical  analyses  of  candidate  PSIs.  ’  ’  These 
indicators  show  good  face  and  construct  validity  and  specificity.'  ’  '  Since  the 
purpose  of  the  hospital-level  PSIs  is  to  identify  instances  where  a  complication  of 
care  occurs  during  a  given  hospital  stay,  PSI  cases  include  only  those  in  which  a 
secondary  diagnosis  code — rather  than  the  principal  diagnosis — flags  a  potential 
patient  safety  event.  Of  the  20  accepted  hospital-level  PSIs,  8  are  for  surgical 
discharges,  8  are  for  either  medical  or  surgical  discharges,  and  4  are  for  obstetric 
discharges.  Because  this  study  compares  PSIs  from  Department  of  Veterans 
Affairs  (VA)  discharge  data  to  non-VA  discharge  data,  we  exclude  the  four 
obstetric  PSIs  here  as  they  are  not  relevant  to  the  VA.  In  this  study,  we  use 
Version  2.1  of  the  AHRQ  PSI  software,  released  March  2003.  Table  1  contains 
the  definitions  of  the  numerators,  denominators,  and  exclusion  criteria  for  the  16 
hospital-level  PSIs  used  in  this  study. 

Use  of  administrative  data 

Compared  to  other  methods  of  detecting  patient  safety  events  (e.g.,  error 
reporting  systems  and  medical  records)5' 17  the  PSIs  offer  several  advantages.  PSIs 
capitalize  on  the  unique  attributes  of  hospital  discharge  administrative  data,  are 
relatively  inexpensive  to  use,  readily  available,  computer  readable,  and  typically 
encompass  large  populations,  thereby  facilitating  population-level  assessments 
based  on  calculation  of  event  rates.7' 32  34  Despite  extensive  empirical  evaluation 
and  clinical  review,  ’  ’  ’  concerns  similar  to  those  raised  about  the  use  of 
administrative-data-based  algorithms  for  identifying  substandard  care  have 
surfaced  in  response  to  the  development  of  the  PSIs.  ~  A  recent  publication 
linking  PSIs  with  increased  mortality,  length  of  stay,  and  charges33  generated 
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considerable  debate  about  the  usefulness  of  the  PSIs  as  a  measure  of  hospital- 
acquired  injuries.  Notwithstanding  such  controversy,  the  development  of  the 


Table  1.  Definitions  of  accepted  hospital-level  AHRQ  Patient  Safety  Indicators  (excludes 
obstetric  and  birth  trauma  indicators) 


Indicator 

Numerator 

Denominator 

Complications 
of  anesthesia 

Discharge  with 
codes  for 
anesthesia 
complications  in 
any  secondary 
diagnosis  field 

All  surgical  discharges.  Exclude  patients  with  codes  for 
poisoning  due  to  anesthetics  and  any  diagnosis  code  for 
active  drug  dependence,  active  nondependent  abuse  of 
drugs,  or  self-inflicted  injury. 

Death  in  low 
mortality  DRGs 

Discharges  with 
disposition  of 
“deceased” 

Patients  in  DRGs  with  less  than  0.5%  mortality  rate,  based 
on  NIS  1997  data.  Exclude  patients  with  any  code  for 
trauma,  immuocompromised  state,  or  cancer. 

Decubitus 

ulcer 

Discharges  with 
decubitus  ulcer 
in  any 
secondary 
diagnosis  field 

All  medical  and  surgical  discharges.  Include  only  patients 
with  LOS  >  4  days.  Exclude  patients  in  MDC  9  or  patients 
with  any  diagnosis  of  hemiplagia,  paraplegia,  quadriplegia. 
Exclude  patients  admitted  from  a  long-term  care  facility. 

Failure  to 

rescue 

All  discharges 
with  disposition 
of  “deceased” 

Discharges  with  potential  complications  of  care  used  in 
failure  to  rescue  definition  (i.e. ,  pneumonia,  DVT/PE, 
sepsis,  acute  renal  failure,  shock/cardiac  arrest,  or  Gl 
hemorrhage/acute  ulcer).  Exclusion  criteria  specific  to  each 
diagnosis.  Also  exclude  patients  transferred  to  or  from 
acute  care  facility;  age  75  and  older;  or  admitted  from  long¬ 
term  care  facility. 

Foreign  body 
left  in  during 
procedure 

Discharges  with 
codes  for  foreign 
body  left  in 
during 

procedure  in  any 
secondary 
diagnosis  field 

All  medical  and  surgical  discharges. 

Iatrogenic 

pneumothorax 

Discharges  with 
ICD-9-CM  codes 
of  512.1  in  any 
secondary 
diagnosis  field 

All  surgical  and  medical  discharges.  Exclude  patients  with 
any  diagnosis  of  trauma  or  any  code  indicating  thoracic 
surgery  or  lung  or  pleural  biopsy  or  cardiac  surgery. 

Infection  due  to 
medical  care 

Discharges  with 
ICD-9-CM  code 
of  999.3  or 

996.62  in  any 
secondary 
diagnosis  field 

All  surgical  and  medical  discharges.  Exclude  patients  with 
any  code  for  immunocompromised  state  or  cancer. 

In-hospital  hip 
fracture 

Discharges  with 
code  for  hip 
fracture  in  any 
secondary 
diagnosis  field 

All  surgical  discharges.  Excludes  patients  who  have 
musculoskeletal  and  connective  tissue  disease  (MDC  8); 
or  with  principal  diagnosis  codes  for  seizure,  syncope, 
stroke,  coma,  cardiac  arrest,  poisoning,  trauma,  delirium 
and  other  psychoses,  or  anoxic  brain  injury;  or  with  any 
diagnosis  of  metastatic  cancer,  lymphoid  malignancy,  bone 
malignancy  or  self-inflicted  injury. 
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Table  1.  Definitions  of  accepted  hospital-level  AHRQ  Patient  Safety  Indicators  (excludes 
obstetric  and  birth  trauma  indicators),  cont. 


Indicator 

Numerator 

Denominator 

Postoperative 
hemorrhage  or 
hematoma 

Discharges  with  codes 
for  postoperative 
hemorrhage  or 
hematoma  in  any 
secondary  diagnosis 
field  AND  code  for 
postoperative  control 
of  hemorrhage  or 
hematoma  in  any 
secondary  procedure 
code  field.  Code  for 
postoperative  control 
of  hemorrhage  or 
hematoma  must  occur 
on  the  same  day  or 
after  the  principal 
procedure. 

All  surgical  discharges. 

Postoperative 
physiologic 
and  metabolic 
derangement 

Discharges  with  codes 
for  physiologic  and 
metabolic 

derangements  in  any 
secondary  diagnosis 
field 

All  elective  surgical  discharges.  Exclude  patients 
with  both  a  diagnosis  code  of  ketoacidosis, 
hyperosmolarity  or  other  coma  (subgroups  of 
physiologic  and  metabolic  derangements  coding) 

AND  a  principal  diagnosis  of  diabetes;  exclude 
patients  with  both  a  secondary  diagnosis  code  for 
acute  renal  failure  (subgroup  of  physiologic  and 
metabolic  derangements  coding)  AND  a  principal 
diagnosis  of  acute  myocardial  infarction,  cardiac 
arrhythmia,  cardiac  arrest,  shock,  hemorrhage  or 
gastrointestinal  hemorrhage. 

Postoperative 

respiratory 

failure 

Discharges  with  ICD- 
9-CM  codes  for  acute 
respiratory  failure 
(518.81)  in  any 
secondary  diagnosis 
field 

All  elective  surgical  discharges.  Exclude  patients 
with  respiratory  or  circulatory  diseases  (MDC  4  and 
MDC  5). 

Postoperative 
pulmonary 
embolism  or 
deep  vein 
thrombosis 

Discharges  with  codes 
for  deep  vein 
thrombosis  or 
pulmonary  embolism 
in  any  secondary 
diagnosis  field 

All  surgical  discharges.  Exclude  patients  with  a 
principal  diagnosis  of  deep  vein  thrombosis,  patients 
with  secondary  procedure  code  38.7  when  this 
procedure  occurs  on  the  day  of  or  before  the  day  of 
principal  procedure. 

Postoperative 

sepsis 

Discharges  with  code 
for  septicemia  in  any 
secondary  diagnosis 
field 

All  elective  surgical  discharges.  Exclude  patients 
with  a  principal  diagnosis  of  infection,  or  any  code  for 
immuncompromised  state,  or  cancer.  Include  only 
patients  with  a  length  of  stay  of  more  than  3  days. 

Accidental 
puncture  or 
laceration 

Discharges  with  code 
denoting  technical 
difficulty  (e.g., 
accidental  cut, 
puncture,  perforation 
or  laceration  during  a 
procedure)  in  any 
secondary  diagnosis 
field 

All  medical  and  surgical  discharges. 
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Table  1.  Definitions  of  accepted  hospital-level  AHRQ  Patient  Safety  Indicators  (excludes 
obstetric  and  birth  trauma  indicators),  cont. 


Indicator 

Numerator 

Denominator 

Transfusion 

reaction 

Discharges  with  codes 
for  transfusion  reaction 
in  any  secondary 
diagnosis  field  per  100 
discharges. 

All  medical  and  surgical  discharges. 

Postoperative 

wound 

dehiscence 

Discharges  with  ICD- 
9-CM  codes  for 
reclosure  of 
postoperative 
disruption  of 
abdominal  wall  (54.61) 
in  any  secondary 
procedure  field 

All  abdominopelvic  surgical  discharges. 

PSIs  has  opened  up  new  opportunities  for  screening  potential  patient  safety  events 
and  paved  the  way  for  implementing  patient  safety  initiatives  and  benchmarking 

3 1  32  34 

hospital  performance.  ’  ’ 

Research  objectives 

The  purpose  of  this  study  is  to  develop  and  test  methods  for  applying  the  PSIs 
to  hospital  discharge  data  from  the  Department  of  Veterans  Affairs  (VA)  and  for 
comparing  VA  with  non-VA  PSI  rates.  Because  the  PSIs  were  developed  and 
tested  using  computerized  hospital  discharge  abstracts  from  AHRQ’s  Healthcare 
Cost  and  Utilization  Project  (HCUP),  PSI  definitions  are  based  on  a  core  set  of 
variables  available  from  standardized  hospital  discharge  abstracts.  The  abstracts 
are  formatted  using  clinical  and  nonclinical  data  elements  from  the  1992  Unifonn 
Bill  (UB-92)  hospital  claims,  considered  the  institutional  claim  standard.7 
However,  unlike  most  State-level  hospital  administrative  databases,  which  contain 
standardized  discharge  abstracts,  VA  databases  have  evolved  using  distinctive 
formatting  structure  and  data  element  definitions.  Furthermore,  VA  hospital 
discharge  data  contain  both  acute  and  nonacute  care,  whereas  HCUP  data  contain 
information  only  from  acute  care  hospitals.  Therefore,  it  is  necessary  to  modify 
some  VA  data  elements  to  provide  the  appropriate  inputs  required  by  the  PSI 
algorithms.  Such  differences  in  data  elements  and  structure  between  the  VA  and 
non-VA  setting  (as  well  as  across  other  health  care  systems)  could  affect 
comparisons  of  PSI  event  rates. 

In  this  paper,  we  describe  the  modifications  we  made  to  VA  file  structure  and 
data  elements  to  (1)  generate  valid  indicator  rates  using  PSI  software  on  VA  data, 
and  (2)  compare  PSI  rates  between  the  VA  and  HCUP  datasets.  Our  goal  is  to 
present  what  we  have  learned,  thereby  facilitating  the  work  of  other  researchers 
and  practitioners  who  wish  to  use  the  PSIs,  particularly  for  comparison  across 
systems  where  there  are  differences  in  the  nature  and  structure  of  the  data. 
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Methods 

VA  patient  treatment  file 

VA  administrative  databases  contain  diagnostic,  demographic,  and  utilization 
information  on  all  veterans  who  receive  health  care  services  in  the  VA.  The  unit 
of  analysis  is  the  hospitalization,  but  since  the  patient  has  a  unique  identifier, 
these  hospitalizations  can  be  linked  by  patient  across  datasets  and  fiscal  years  and 
aggregated  at  the  patient  level.  Both  acute  and  nonacute  hospitalization  data  on 
veterans  discharged  from  VA  inpatient  facilities  are  contained  in  the  Patient 
Treatment  File  (PTF).  Currently,  there  are  140  VA  hospitals  nationwide  that 
provide  information  to  the  PTF.39  The  PTF  is  comprised  of  four  subfiles,  referred 
to  as  the  Main,  Bedsection,  Procedure,  and  Surgery  files.39,40 

The  PTF  Main  file  contains  demographic  (e.g.,  age,  sex,  date  of  birth), 
diagnostic  (one  principal,  one  primary,  and  up  to  nine  secondary  ICD-9-CM 
diagnosis  codes),  and  summary  information  related  to  each  episode  of  care  (e.g., 
facility  identifier,  dates  of  admission  and  discharge,  setting  of  care  within  facility, 
discharge  status).  The  PTF  Bedsection  file  (i.e.,  a  patient’s  stay  under  a  specified 
treating  physician’s  specialty  service)  contains  one  primary  and  up  to  four 
secondary  bedsection  diagnoses,  as  well  as  length  of  stay  information  for  each 
stay  in  a  physician’s  specialty  service.  A  patient  may  have  several  bedsection 
records  for  one  hospitalization.  The  PTF  Procedure  file  includes  the  ICD-9-CM 
procedure  code,  date,  time,  and  place  of  all  procedures  during  the  inpatient  stay; 
similarly,  the  PTF  Surgery  file  contains  data  on  each  hospitalization’s  ICD-9-CM 
surgery  codes  and  surgical  specialty. 

AHRQ  HCUP  State  inpatient  databases 

HCUP  is  a  Federal-State-private  sector  collaboration  sponsored  by  AHRQ 
that  collects  hospital  discharge  abstract  data  for  research  purposes.  Currently,  36 
States  participate  in  HCUP,  with  the  data  comprising  approximately  90  percent  of 
all  hospital  discharges  in  the  United  States.41,42  Each  HCUP  inpatient  record 
summarizes  one  hospital  discharge.  Statewide  hospital  data  collection  programs 
voluntarily  submit  data  to  HCUP,  where  the  data  are  converted  into  a  uniform 
format  to  facilitate  multistate  analyses.  The  State  databases  differ  in  how  many 
diagnoses  and  procedures  can  be  recorded  for  a  hospital  stay:  some  States  provide 
up  to  30  diagnosis  and  procedure  fields,  while  some  have  as  few  as  10.  The 
uniformly  fonnatted  HCUP  discharge  data  are  available  as  State  Inpatient 
Databases  (SID),  which  include  all  data  from  hospitals  in  participating  States.  A 
sample  is  drawn  from  the  SID,  approximating  a  20  percent  sample  of  all  U.S. 
hospitals;  this  database  is  called  the  Nationwide  Inpatient  Sample  (NIS).  For  this 
analysis,  we  employ  a  previously  published  study  that  applied  the  PSIs  to  the  NIS. 
A  subsequent  study  will  compare  PSI  rates  in  the  VA  to  data  from  the  SID. 
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Modifying  VA  data  for  PSI  software 

Data  required  for  PSI  software 

PSI  algorithms  link  diagnosis  and  procedure  codes  with  other  information 
contained  in  standardized  hospital  discharge  data,  such  as  diagnosis  related  group 
(DRG)  and  admission  type,  to  generate  PSI  event  rates.  The  required  data 
elements  are  age,  sex,  race,  hospital  identification  number,  disposition  of  patient, 
admission  type,  admission  source,  length  of  stay  (LOS),  DRG,  major  diagnostic 
category  (MDC),  ICD-9-CM  principal  and  secondary  diagnosis  codes,  ICD-9-CM 
principal  and  secondary  procedure  codes,  number  of  diagnoses,  number  of 
procedures,  and  days  from  admission  to  procedure.30  Input  data  files  must  be  in 
Statistical  Analytical  System  (SAS)  or  Statistical  Package  for  the  Social  Sciences 
(SPSS)  for  running  the  PSI  software. 

Modifications  to  VA  data  elements 

To  create  a  database  for  fiscal  year  (FY)  2001,  we  used  all  PTF  files  with 
veterans-only  hospitalization  discharge  dates  between  October  1,  2000,  and 
September  30,  2001.  Admission  date  could  precede  October  1,  2000;  therefore, 
some  lengths  of  stay  exceed  365  days.  Some  required  PSI  data  elements  were  in 
the  VA  files  and  needed  minimal  or  no  recoding:  age,  sex,  race,  LOS,  hospital 
identifier,  disposition  of  patient,  DRG,  MDC,  and  principal  and  secondary 
diagnoses.  Other  data  were  available  but  needed  to  be  calculated  or  modified: 
principal  procedure,  admission  type,  and  days  from  admission  to  procedure. 
Finally,  one  data  element  was  completely  missing — admission  source — and 
needed  to  be  constructed  based  on  other  existing  data  elements.  The  following  is  a 
discussion  of  those  data  elements  that  required  some  modification. 

Principal  procedure.  The  definitions  of  three  PSIs  (postoperative 
hemorrhage  or  hematoma,  postoperative  pulmonary  embolism  or  deep  vein 
thrombosis,  and  postoperative  wound  dehiscence)  include  the  principal  procedure, 
and  several  PSI  definitions  include  secondary  (i.e.,  any  but  the  principal) 
procedure.  Because  VA  files  do  not  indicate  principal  procedure,  we  developed  an 
algorithm  to  do  this.  The  chronological  first  procedure  in  the  Surgery  file — or  the 
first  in  the  Procedure  file  if  there  is  no  Surgery  file — would  seem  a  logical 
candidate  for  principal.  However,  an  attribute  of  the  level  of  detail  in  the  Surgery 
and  Procedure  files  is  that  relatively  minor  procedures  (e.g.,  administering 
oxygen)  as  well  as  more  major  but  nonsurgical  procedures  (e.g.,  cardiac 
catheterization)  may  precede  major  surgeries  (e.g.,  bypass  surgery).  Designating 
these  first  minor  or  nonsurgical  procedures  “principal”  would  contradict  the  logic 
of  these  PSIs:  the  principal  procedure — defined  as  the  procedure  performed  for 
definitive  treatment  or  that  is  most  related  to  the  principal  diagnosis — should  be  a 
clinically  plausible  cause  of  the  potential  safety  event. 

Because  output  from  the  DRG  grouping  software  indicates  whether  a  given 
procedure  is  typically  perfonned  in  the  operating  room  (OR),  we  elected  to  use  a 
list  of  all  ICD-9-CM  procedure  codes  designated  as  “valid  OR  procedures”  by  the 
DRG  grouper  to  identify  “true”  surgeries  from  the  Surgery  and  Procedure  files. 
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Using  this  list  as  the  criterion,  nearly  3  percent  of  FY2001  VA  hospitalizations 
with  surgical  DRGs  did  not  include  valid  OR  procedures.  These  represented 
surgical  DRGs  with  particular  combinations  of  non-OR  procedures  (e.g.,  certain 
pacemaker  insertions  or  temporary  tracheostomies).  Our  clinician  team  elected  to 
eliminate  cases  without  valid  OR  procedures  from  consideration  for  surgical  PSIs, 
but  to  include  them  in  the  risk  pools  for  PSIs  that  are  not  limited  to  surgical 
discharges.  We  then  modified  our  algorithm  to  identify  the  principal  procedure  as 
the  first  chronologically  valid  OR  procedure  from  either  the  Surgery  or  the 
Procedure  file. 

Admission  type.  The  PSIs  detect  potential  events  primarily  related  to  elective 
surgery.  Three  PSIs — postoperative  physiological  and  metabolic  derangements, 
postoperative  pulmonary  compromise,  and  postoperative  sepsis — exclude 
nonelective  hospitalizations  from  the  PSI  denominator.  This  reflects  the  logic  that 
any  complications  occurring  in  patients  admitted  for  nonelective  surgeries,  or 
urgent/emergent  conditions,  are  less  likely  to  be  preventable,  given  the  need  for 
immediate  care.  To  exclude  nonelective  admissions,  the  PSI  software  searches  for 
admission  type  (emergent/urgent,  elective);  if  admission  type  is  missing,  the  PSI 
software  uses  the  admission  source  (e.g.,  emergency  department,  transfer  from 
hospital,  long-term  care)  to  determine  admission  type.  The  VA  PTF  lacks  an 
admission  type  field  and,  although  there  is  a  field  for  admission  source,  there  is  no 
code  for  one  important  source,  the  emergency  department. 

We  developed  an  algorithm  that  uses  DRG,  admission  date  and  time,  and 
principal  procedure  date  and  time,  to  distinguish  between  elective  and  nonelective 
cases.  The  algorithm  first  screens  out  urgent/emergent  admissions  using  a  list  of 
DRG  codes.  This  list  of  nonelective  DRGs,  originally  developed  for  use  with 
California  hospital  data  that  lacked  admission  type,  included  mostly  trauma  DRG 
codes;  our  clinicians  added  codes  for  other  surgeries  also  considered 
urgent/emergent  (e.g.,  appendectomy).  The  algorithm  then  determines  the  time 
elapsed  between  admission  and  principal  procedure.  To  be  consistent  with  the 
Complications  Screening  Protocol,  we  designated  surgery  performed  on  the  day 
of  or  the  day  after  admission  as  elective,  with  surgery  perfonned  on  or  after  the 
third  day  as  nonelective.  Urgent/emergent  surgeries,  such  as  appendectomy 
upon  admission,  are  excluded  by  the  trauma  DRG  screen.  Other  urgent/emergent 
cases  could  involve  surgery  after  the  second  day  of  hospitalization,  allowing  time 
to  stabilize  the  patient  and  perform  diagnostic  testing.  Because  VA  data  include 
admission  and  procedure  times,  as  well  as  dates,  we  refined  the  algorithm  to 
exclude  (i.e.,  designate  urgent/emergent)  weekend  and  evening  (5  p.m.  to  5  a.m.) 
admissions  and  hospitalizations  where  the  principal  procedure  occurs  on  a 
weekend  or  in  the  evening.  The  remaining  hospitalizations  are  considered 
elective. 
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Modifying  VA  data  for  comparison  with  non-VA  data 

Modifications  to  VA  file  structure 

Compared  to  HCUP  data,  the  four  VA  inpatient  data  files  appear  to  be  a  richer 
source  of  patient-level  and  hospitalization-level  information.  Examples  of  this 
include  the  ability  to  link  multiple  hospital  stays  associated  with  one  patient;  the 
availability  of  bedsection  data;  more  diagnosis  and  procedure  codes;  and 
admission,  discharge,  and  procedure  times.  However,  to  assure  a  meaningful 
comparison  of  PSI  rates  between  the  VA  and  HCUP,  we  modified  the  PTF  files  to 
“equalize”  information  between  the  two  data  sources,  as  described  below. 

Acute-only  hospitalizations.  Because  VA  hospitalization  records  include 
nonacute  and  acute  care,  while  HCUP  records  contain  acute  care  only,  we 
eliminated  nonacute  care  from  the  VA  records  using  the  VA  Health  Economics 
Resource  Center’s  (HERC)  definitions  of  acute  and  nonacute  bedsections. 43,44 
HERC  defines  the  following  bedsections  as  nonacute:  rehabilitation,  spinal  cord, 
substance  abuse,  domiciliary,  long-term  care,  blind  rehabilitation,  psychiatry, 
intennediate  medicine,  and  psychosocial  residential  rehabilitation  treatment 
program  (PRRTP).  We  separated  the  PTF  Main  file  hospitalizations  into  three 
groups  by  examining  the  bedsections  associated  with  each  hospitalization:  (1) 
“pure”  acute  hospitalizations  (all  bedsections  in  the  hospitalization  are  acute),  (2) 
“pure”  nonacute  hospitalizations  (all  bedsections  in  the  hospitalization  are 
nonacute),  and  (3)  “mixed”  hospitalizations  (one  hospitalization  includes  both 
acute  and  nonacute  bedsections). 

Pure  acute  hospitalizations  were  left  unmodified.  Pure  nonacute 
hospitalizations  were  excluded  from  the  analytical  database.  We  eliminated 
nonacute  bedsections  from  mixed  hospitalizations,  and  created  entirely  new  acute 
hospitalizations  from  the  remaining  sets  of  contiguous  acute  bedsections.  More 
than  one  new  hospitalization  may  be  created  from  a  mixed  hospitalization  (e.g.,  a 
mixed  hospitalization  with  one  nonacute  bedsection  between  acute  bedsections 
becomes  two  acute  hospitalizations).  Elimination  of  nonacute  bedsections  from 
mixed  hospitalizations  resulted  in  some  acute  hospitalizations  with  new  discharge 
dates  prior  to  FY2001;  those  hospitalizations  were  eliminated. 

We  thus  created  a  new  database  from  the  four  existing  PTF  files,  with  the 
hospital  discharge  as  the  unit  of  analysis.  The  resulting  hospital  discharge 
summary  file,  constructed  of  data  aggregated  from  the  PTF  Main,  Bedsection, 
Procedure,  and  Surgery  files,  contained  only  acute  care,  and  was  similar  in 
structure  to  the  standardized  hospital  discharge  abstract  found  in  HCUP  data. 
Although  we  risked  losing  some  potentially  useful  information  from  the  VA  by 
making  these  changes,  this  risk  was  minimized  because,  as  we  describe  in  the 
next  section,  we  selected  the  unique  data  elements  from  each  of  the  files  for  the 
summary  discharge  file. 

Creation  of  new  summary  file.  Principal  procedure:  The  principal 
procedure  algorithm  is  essentially  the  same  for  the  new  aggregated 
hospitalizations  as  for  the  pure  acute  hospitalizations:  the  first,  chronologically 
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valid  OR  procedure  from  the  Surgery  or  Procedure  file  is  selected,  as  long  as  the 
date  falls  between  the  new  admission  and  new  discharge  dates. 

Diagnosis:  The  distinction  between  principal  and  secondary  diagnoses  is 
central  to  several  PSI  definitions.  The  diagnosis  related  to  the  reason  for 
admission  is  principal.  In  VA  PTF  hospitalization  records,  the  Main  file  principal 
diagnosis  is  consistent  with  this  definition  and  can  be  used  as  principal  diagnosis 
for  the  PSIs.  However,  because  our  method  of  aggregating  acute  hospitalizations 
eliminated  nonacute  bedsections,  and  some  of  those  nonacute  bedsections 
preceded  the  acute  portion  of  the  hospitalization,  the  principal  diagnosis  from  the 
Main  file  may  have  originated  from  the  nonacute  bedsection  and  may  not  have 
applied  to  the  acute  hospitalization.  Therefore,  for  the  newly  aggregated 
hospitalizations,  we  created  the  following  rules  for  determining  principal 
diagnosis:  (1)  If  the  new  and  original  admission  dates  are  the  same,  use  the  Main 
file  principal  diagnosis;  (2)  if  the  dates  are  different,  choose  the  primary  diagnosis 
(diagnosis  responsible  for  the  bedsection  stay)  from  the  first  bedsection  as  a  proxy 
for  principal  diagnosis  (bedsection  files  lack  a  principal  diagnosis  field). 

DRG:  The  DRG  grouper  designates  each  DRG  as  either  medical  or  surgical; 
PSI  software  uses  DRGs  to  classify  a  discharge  as  surgical  or  medical.  The 
problem  and  solution  for  identifying  the  correct  DRG  for  the  new  summary  file 
parallel  those  for  principal  diagnosis.  While  the  DRG  from  the  PTF  Main  file  is 
appropriate  for  pure  acute  hospitalizations  when  no  nonacute  bedsections  were 
discarded,  in  the  newly  aggregated  acute  hospitalizations,  a  DRG  must  be  selected 
from  among  the  DRGs  associated  with  each  of  the  one  or  more  remaining  acute 
bedsections.  The  DRG  for  each  bedsection  is  assigned  based  on  the  diagnoses  and 
procedures  associated  with  that  bedsection.  Since  the  DRG  grouper  assigns  a 
weight  to  each  DRG  for  costing  purposes,  our  solution  was  to  select  the 
bedsection  DRG  with  the  highest  weight  for  our  new  hospitalization  summary 
file.  Our  rationale  was  that  in  the  private  sector,  this  DRG  would  be  selected  to 
maximize  reimbursement.  This  also  follows  the  HERC  method,  which  selects  the 
highest  weighted  DRG  as  the  discharge  DRG  among  the  different  bedsection 
DRGs. 

Admission  type:  The  admission-type  algorithm  described  earlier,  developed 
for  use  with  the  original  VA  PTF  Main  file  data,  could  not  be  used  in  its  entirety 
for  the  HCUP  comparison;  HCUP  data  do  not  include  admission  times  or 
procedure  times.  Therefore,  we  eliminated  admission  time  and  principal 
procedure  times  from  the  screens  used  by  the  algorithm. 


Results 

Effects  of  modifying  file  structure 

Table  2  compares  the  modified  VA  data  to  the  original  VA  data  files  and 
structure.  Compared  to  the  complete  PTF  database  (left-hand  column),  the 
modified  acute-only  file  (right-hand  column)  contains  substantially  fewer 
hospitalizations,  as  expected,  because  we  eliminated  nonacute  bedsections.  These 
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Table  2.  VA  hospitalizations:  FY2001  discharges,  veterans  only 


Demographic  information 

Complete  VA  PTF  Main 
file 

(acute  and  nonacute) 

Modified  acute-only 
file 

Hospitalizations:* 

Total1' 

561,229 

439,537 

Medical  (DRG=medical) 

459,653 

340,971 

Surgical  (DRG=surgical) 

101,548 

98,550 

Length  of  stay  (days): 

Mean 

11.6 

7.2 

Median 

6.0 

5.0 

Maximum 

20,229 

2,228 

Age  at  admission  (years): 

Mean 

62.1 

64.8 

Median 

63.0 

66.0 

Minimum 

18 

19 

Maximum 

112 

112 

Age  groups  for  age  at  admission 
(years): 

18-39 

4.5% 

2.6% 

40-64 

49.0% 

43.1% 

65-74 

23.0% 

26.7% 

75-84 

20.6% 

24.2% 

85+ 

2.8% 

3.3% 

Sex: 

Male 

96.5% 

97.0% 

Female 

3.5% 

3.0% 

Ethnicity: 

White 

68.2% 

69.6% 

Black 

20.1% 

18.2% 

Hispanic 

5.0% 

5.2% 

Asian/Pacific  Island 

0.3% 

0.3% 

Native  American 

0.4% 

0.4% 

Other 

6.0% 

6.3% 

Death: 

Died  in  hospital 

3.6% 

3.9% 

*  Mean  number  of  hospitalizations  per  patient  per  year  is  1 .58  in  the  complete  file  and  1 .53  in  the 
acute-only  file. 

f  Medical  and  Surgical  do  not  sum  to  total  because  there  are  28  hospitalizations  with  DRGs  that 
are  ungroupable  or  unrelated  to  the  VA  population  (e.g.  newborn,  premature)  in  the  VA  file  and 
16  in  the  acute  file. 
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modifications  reduced  the  total  number  of  hospitalizations  22  percent,  from 
561,229  to  439,537.  The  reduction  in  total  records  resulting  from  the  elimination 
of  pure  nonacute  hospitalizations  was  balanced  somewhat  by  the  creation  of  new 
acute  hospitalizations  from  the  mixed  acute/nonacute  records.  Reconfiguring  the 
mixed  acute/nonacute  hospitalizations  (about  8  percent  of  the  total)  generated 
from  one  to  eight  new  acute  hospitalizations  from  each  mixed  hospitalization. 
Similarly,  median  and  mean  lengths  of  stay  decreased  by  1  and  approximately  4 
days,  respectively,  while  median  and  mean  age  increased  by  about  2  and  3  years, 
respectively.  There  were  minor  differences  in  sociodemographic  characteristics 
between  the  two  sets  of  hospitalizations. 

Table  3  depicts  the  numerators,  denominators,  and  unadjusted  rates  for  each 
of  the  PSIs,  using  both  the  original  VA  database  and  the  aggregated  acute-only 
database.  Both  the  numerators  and  denominators  decreased  for  all  PSIs;  however, 
there  was  a  slight  increase  in  the  rates  for  some  PSIs  and  a  slight  decrease  for 
others.  For  example,  rates  of  infection  due  to  medical  care  increased  from  1.91  to 
2.37  per  1,000  discharges,  while  rates  of  postoperative  hip  fracture  decreased 
from  1.90  to  1.14  per  1,000  discharges. 

Effects  of  modifying  admission  type  definition 

The  VA-only  database  has  a  smaller  number  of  elective  admissions  because 
hospitalizations  with  evening  and  nighttime  admissions  (after  5  p.m.)  or  with 
principal  procedure  occurring  after  hours  (after  5  p.m.)  are  also  designated 
nonelective;  whereas,  in  the  VA-HCUP  comparison  database,  those  “after-hours” 
cases  remain  in  the  pool  of  hospitalizations  designated  elective.  In  the  VA-only 
database,  elective  admissions  make  up  43  percent  of  all  surgical  hospitalizations 
with  one  or  more  valid  OR  procedures;  in  the  VA-HCUP  comparison  database, 
this  rate  of  elective  admissions  rises  to  61  percent.  Table  4  compares  PSI  rates  for 
the  three  indicators  for  which  the  risk  pool  is  elective  admissions  only.  Removing 
admission  and  procedure  times  from  the  algorithm  (i.e.,  more  admissions 
designated  “elective”)  increased  the  rates  of  all  three  PSIs,  suggesting  that  the 
definition  that  includes  admission  and  procedure  times  may  have  better  specificity 
(i.e.,  capturing  the  cases  that  are  “true”  PSIs). 

PSI  rates 

Table  5  compares  the  PSI  rates  that  were  calculated  using  the  VA  aggregated- 
acute  database  with  rates  presented  in  a  previously  published  nationwide  study. 

As  noted  in  the  table,  some  PSI  rates  are  not  fully  comparable  between  the  two 
studies  due  to  changes  in  PSI  inclusion/exclusion  criteria  between  the  earlier 
version  of  the  PSI  software  used  by  Romano  et  al.  and  Version  2. 1  used  in  this 
study.  The  nationwide  rates  in  Table  5  are  based  on  NIS  data  from  AHRQ,  as 
reported  by  Romano  et  al.  Therefore,  the  original  admission-type  field,  and  not 
the  admission-type  algorithm  described  above,  was  used  to  determine  elective 
admissions  in  the  Romano  et  al.  study.  Overall,  VA  and  non-VA  rates  are  similar. 
However,  for  a  few  indicators,  rates  differ  substantially.  For  example,  for  death  in 
low-mortality  DRGs  among  males  age  65-74  years,  the  VA  rate  is  4.28  per 
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thousand,  while  the  non-VA  rate  is  2.02;  for  postoperative  sepsis  in  this  age 
group,  the  VA  rate  is  6.48  per  thousand  while  the  non-VA  rate  is  12.32. 


Discussion 

The  purpose  of  this  study  was  to  develop  and  test  methods  for  applying  PSIs 
to  VA  hospital  discharge  data  and  for  comparing  VA  with  non-VA  PSI  rates.  The 
VA  inpatient  administrative  database  was  modified  to  bridge  apparent  differences 
between  the  VA  database  and  the  HCUP  databases.  First,  modifications  were 
needed  because  certain  elements  required  by  the  PSI  software  were  missing  or 
defined  differently  in  the  VA  data.  While  some  modifications  had  little  or  no 
impact  on  database  characteristics  or  PSI  rates,  others,  such  as  designating 
principal  procedure  and  admission  type,  had  substantial  impact.  Differences 
among  the  rules  that  we  considered  for  designating  principal  procedure,  and 
thereby  for  designating  all  other  procedures  as  secondary,  affected  both  the 
numerators  and  denominators  for  several  PSIs.  Similarly,  the  choice  of  algorithm 
to  designate  elective  admissions  substantially  affected  both  the  rates  of  elective 
admissions  and  those  of  three  PSIs. 

Second,  the  greatest  difference  between  VA  and  non-VA  databases  that  might 
affect  our  results  was  the  inclusion  of  nonacute  care  in  the  VA  data.  For 
comparison  of  VA  and  non-VA  PSI  rates,  it  was  necessary  to  eliminate  nonacute 
hospitalizations  as  well  as  the  nonacute  portions  of  mixed  acute/nonacute 
hospitalizations  and  then  reaggregate  the  remaining  acute  portions  of  the  mixed 
hospitalizations.  The  altered  file  structures  gave  new  characteristics  to  the 
reaggregated  hospitalizations,  requiring  us  to  make  further  changes  to  certain  data 
elements  in  the  affected  files.  This  included  changes  to  length  of  stay,  principal 
procedure,  principal  diagnosis,  and  DRG.  This  “equalization”  of  the  VA  and 
HCUP  databases  achieved  comparability  of  PSI  rates  between  the  two  sources; 
however,  it  also  created  a  less  rich  and  “true”  representation  of  veterans’  inpatient 
care. 

Finally,  there  were  substantial  differences  in  some  PSI  rates  between  the 
complete  and  the  acute-only  VA  databases.  This  demonstrates  the  sensitivity  of 
PSI  rates  to  changes  in  data  aggregation  and  to  inclusion  of  nonacute  care.  It  also 
indicates  that  PSIs  can  identify  potential  safety  events  in  the  nonacute  setting  as 
well  as  the  acute  care  setting,  and  suggests  that  certain  PSI  events  (e.g., 
postoperative  hip  fractures)  are  relatively  more  likely  than  others  to  occur  in  the 
course  of  nonacute  care.  In  addition,  while  a  portion  of  the  difference  in  the  rate 
between  the  original  VA  data  and  the  acute-only  data  is  attributable  directly  to  the 
elimination  of  nonacute  hospital  stays  and  portions  of  hospital  stays,  a  portion  is 
also  an  artifact  of  changes  in  principal  procedure,  principal  diagnosis,  DRG,  and 
number  of  procedure  codes  for  mixed  acute  and  nonacute  hospitalizations.  A 
change  in  any  of  these  elements  of  the  hospitalization  record  can  result  in  a  record 
being  excluded  or  included  in  a  PSI  numerator  or  denominator. 
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Table  3.  Unadjusted  FY2001  rates  for  accepted  hospital  level  Patient  Safety  Indicators:  comparison  of  complete  VA  files  with  acute-only  VA  files 


PSIs  and  Data  Comparability 


Acute-only  VA  files  (n  =  430,552)t 

Unadjusted 

rates 

(per  1,000 
discharges 
at  risk) 

0.56 

CO 

CNJ 

CO 

15.41 

155.55 

0.17 

1.17 

2.37 

1.14 

3.24 

1.89 

3.43 

O 

O 

CO 

6.13 

6.41 

2.82 

r^- 

o 

o 

o 

Denominator 

97,482 

55,079 

208,097 

21,318 

430,536 

402,185 

345,442 

CO 

LO 

CD 

97,479 

CNJ 

O 

CO 

o 

31,221 

97,231 

17,293 

20,115 

430,524 

CO 

CO 

LO 

o 

CO 

Numerator 

55 

178 

3,207 

3,316 

73 

469 

817 

CO 

316 

77 

107 

1,262 

106 

129 

1,216 

CO 

Complete  VA  files  (n  =  551,353) 

Unadjusted 

rates 

(per  1,000 
discharges 
at  risk) 

0.57 

2.05 

13.31 

153.41 

0.14 

0.94 

1.91 

1.90 

3.30 

1.96 

3.57 

CO 

CO 

6.23 

6.50 

2.26 

0.005 

Denominator 

100,458 

CO 

CO 

o 

300,402 

CO 

CO 

CNJ 

551,325 

519,866 

457,708 

73,325 

100,455 

CNJ 

oo 

o 

31,405 

CO 

o 

CNJ 

o 

o 

17,501 

20,454 

551,311 

551,325 

Numerator 

57 

CO 

o 

CNJ 

3,998 

3,710 

76 

487 

873 

139 

331 

o 

CO 

112 

r-- 

co 

CO 

109 

CO 

CO 

1,245 

CO 

Patient  Safety  Indicator 

Complications  of  anesthesia 

Death  in  low  mortality  DRGs 

Decubitus  ulcer 

Failure  to  rescue 
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Table  5.  Unadjusted  rates  for  accepted  hospital  level  Patient  Safety  Indicators:  comparison  of  VA  FY2001  (acute-only)  and  HCUP  2000  Nationwide 
Inpatient  Sample 
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PSI  software  now  excludes  cases  with  age>=75  for  this  indicator. 


PSIs  and  Data  Comparability 


Our  results  agree  with  the  literature  that  raises  concerns  over  the  use  of 
administrative  data-based  algorithms  for  detecting  patient  safety  events.15'35 
While  this  approach  to  measuring  patient  safety  has  distinct  advantages  over  other 
methods,  it  carries  the  risk  of  capturing  false  events.  Our  results  suggest  that  other 
health  care  systems  may  face  similar  needs  to  make  modifications  to  their  data  in 
order  to  apply  the  PSI  software.  While  this  study  demonstrated  that  it  is  possible 
to  modify  data  elements  to  achieve  a  high  degree  of  comparability  across  health 
care  systems  with  different  databases,  such  differences  in  data  elements  and 
structure  between  systems  could  still  affect  comparisons  of  PSI  event  rates. 
Nonetheless,  because  we  did  not  examine  risk-adjusted  PSI  rates  in  this  study, 
further  research  is  critical  to  better  our  understanding  of  how  meaningful  these 
differences  really  are. 


Conclusion 

We  have  demonstrated  the  sensitivity  of  PSI  rates  to  differences  in  data  file 
structure  and  to  definitions  and  sources  of  data  elements.  The  consequences  of 
this  sensitivity  are  amplified  by  the  fact  that  PSI  rates  are  inherently  low:  most 
PSI  rates  are  in  the  range  of  one  to  five  per  thousand  hospitalizations.  Therefore, 
differences  in  data  structures  and  algorithms  that  add  or  subtract  just  one  or  a  few 
cases  from  the  numerator  of  a  PSI  for  a  given  population  and  time  period  could 
make  a  meaningful  difference  in  the  overall  PSI  rate. 
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